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ABSTRACT  (Continue  an  toeereo  elde  If  necoeeeey  end  Identify  by  block  number) 

he  purpose  of  this  cost-effectiveness  specification  is  to  facilitate  the  pur¬ 
chase,  monitor,  and  evaluation  of  computer-based  training  systems.  This  stand¬ 
ardized  structure  for  deriving  and  communicating  training  system  costs  and  ef¬ 
fectiveness  is  presented  in  three  volumes,  corresponding  to  the  life  cycle  of  a 
computer-based  training  system:  (1)  development,  (2)  procurement,  and  (3)  oper¬ 
ation  and  maintenance.  The  cost  methodology  focuses  on  identifying  and  quanti¬ 
fying  total  inputs  required  by  the  system  over  its  life  cycle.  Effectiveness 
dimensions  include  ob.iectives-based  achievement  and  time  m 
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ourse  and  end-of-course  criteria.  Other  measures,  such  as  attrition  rates. 
Instructor  ratings,  and  attitude  scales,  are  also  discussed,  x 
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PREFACE 


This  Is  Voluse  II  of  Che  Cost-Effectiveness  Specification  for  Coaputer- 
Based  Training  Systeas.  It  is  part  of  a  three-voluae  set  which  correspond 
to  the  three  phases  of  a  training  systea's  life  cycle  (Voliae  I  -  Developaent; 
Voluae  II  -  Procureaent;  Voluae  III  -  Operations  and  Maintenance).  An 
Executive  Suaaary  docuaent  is  included  which  provides  the  reader  with 
general  guidance  and  instructions  on  how  to  go  through  and  respond  to  the 
various  parts  of  the  specification.  This  Cost-Effectiveness  Specification 
was  developed  for  DARPA  as  part  of  Contract  IMDA903-76-C-0210. 

Dr.  Harold  F.  O'Neil  was  the  Technical  Monitor. 
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Part  A 

SYSTEM  DESCRIPTION 


1 


INSTRUCTIONS 


The  purpose  of  Part  A  is  to  describe  some  of  the  characteristics  of 
the  computer-based  training  system  and  to  identify  the  individuals  who 
are  using  this  specification. 


Print  the  name  or  descriptive  title  of  the  computer-based  training 
system. 


Enter  the  year  which  represents  the  system's  current  life  cycle 
status.  In  this,  the  Procurement  Phase,  indicate  the  year  following 
the  final  year  of  the  Development  Phase. 


© 

© 

© 


Check  the  type  of  hardware  configuration  supporting  the  system. 


Enter  the  maximum  number  of  time-sharing  trainees  pt-r  system. 


List  the  courses  to  be  taught  or  supported  by  the  computer-based 
training  system. 


Estimate  the  number  of  trainees  who  are  to  be  served  by  the  system 
in  each  course,  and  in  all  courses  per  year. 

Print  your  name,  job  title,  and  organization. 
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SYSTEM  DESCRIPTION 


(T)  Computer-Based  Training  System 

System  Life  Cycle 

Phase  II  Procurement 

(T)  Configuration 

Stand  Alone 

(4)  Time-Sharing  Capacity 

t 

r 

f. 
i 

Ir 

I  (i)  Courses  Supported  by  System  (Speedy) 


19 


□  Remote 


[  |  Combination  Stand  Alone  and  Remote 


© 

No.  of  Trainees 
Per  Year 


Total 


Your  Name 


Job  Title 


Organization 
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GENERAL  COSTING  ASSUMPTIONS  AND  DEFINITIONS 


GENERAL  COSTING  ASSUMPTIONS  AND  DEFINITIONS 


PROCUREMENT  PHASE 

Activites  related  to  the  production,  purchase,  and  installation  of 
the  operational  computer-based  training  system.  Included  in  these  acti¬ 
vities  are:  fabrication,  communication,  reproduction,  packaging  and 
shipping,  instructional  preparation,  etc.  Procurement  costs  are  all  costs 
(in-house  and  contractor),  irrespective  of  how  funded,  which  are  necessary 
to  transform  or  copy  the  tested,  prototype  system  into  a  fully  operational 
system  consisting  of  the  hardware,  software,  facilities,  training,  and 
support  necessary  to  initiate  operations.  These  costs  include  the  following 
components: 

•  Equipment.  The  costs  associated  with  purchasing,  manufacturing, 
shipping,  and  installing  the  hardware  (all  parts  and  equipment  necessary 

for  the  system)  as  well  as  costs  necessary  to  process,  assemble,  and  integrate 
the  final  end-items  in  a  workable  system. 

•  Fad! ities.  The  costs  of  real  estate,  construction,  conversion, 
utilities  and  equipment  to  provide  all  facilities  required  to  house  the 
operational  computer-based  training  system. 

•  Softwa re.  The  costs  of  reproduction,  packaging,  shipping,  installa¬ 
tion,  and  operational  checkout  of  computer  programs  supporting  the  training 
system. 

•  Instructional  Preparation.  The^psts  of  training  a  cadre  of  initial 
site  personnel  to  operate  and  maintain  ale  system. 

•  Instructional  Methods/Materials.  The  costs  of  fabrication,  reproduc¬ 
tion,  communication,  packaging  and  shipping  of  all  training  materials 
required  by  the  operational  system. 

•  Acceptance  Test/Management.  The'  costs  of  managing  the  production, 
purchase  and  installation  of  the  system  as  well  as  the  costs  of  acceptance 
testing,  site  checkout/activation,  and  making  engineering  changes  if  necessary. 

•  Other  Direct  Costs.  Any  dJItect  procurement  costs  not  included  in 
the  previous  components. 

COMPUTER-BASED  TRAINING  SYSTEM 

The  application  focus  of  this  costing  specification  is  formal,  school- 
based  training.  The  specification  is  oriented  toward  training  which  is 
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administered,  aided,  or  managed  by  computer  (hence,  computer-based).  The 
system  component  costs  are  dijtributed  across  the  training  system's  life 
cycle  which  consists  of  three  phases:  Development,  Procurement,  Operation 
and  Maintenance. 

CONSTANT  DOLLARS 

In  order  to  make  valid  comparisons  between  alternatives,  the  cost  for 
each  alternative  must  be  stated  in  the  same  terms.  In  Part  D  of  the 
specification  all  costs  are  to  be  adjusted  so  that  they  are  expressed  in 
constant  year  dollars.  Constant  year  dollars  express  all  costs  expended  in 
various  years  in  terms  of  the  general  purchasing  power  of  the  dollar  for  a 
given  base  year.  An  estimate  is  said  to  be  in  constant  dollars  if  costs  for 
all  work,  both  prior,  current,  and  future,  are  adjusted  so  that  they  reflect 
the  level  of  prices  of  the  base  year.  For  purposes  of  this  specification, 
the  base  year  ts  the  calendar  year  in  which  the  costing  analysis  is  performed. 

If  cost  data  or  estimates  are  available  in  other  than  constant  year 
dollars,  constant  year  dollars  are  arrived  at  by  applying  the  appropriate 
adjustment  factors.  Current  dollars  reflect  purchasing  power  current  to 
the  year  in  which  they  are  expended.  Prior  costs  stated  in  current  dollars 
are  the  actual  amounts  paid  out  in  these  years.  Future  costs  stated  in 
current  dollars  are  the  projected  actual  amounts  which  will  be  paid.  Care 
should  be  exercised  to  preclude  the  mixing  of  current  dollars  with  constant 
dollars  in  a  single  display  of  costs.  Any  cost  figures  provided  in  current 
dollars  are  to  be  clearly  identified  as  such. 

DISCOUNTED  RATES 

Discounting  is  a  technique  for  converting  various  cash  flows  occurring 
over  time  to  equivalent  amounts  at  a  common  point  in  time,  considering  the 
time  value  of  money  to  facilitate  a  valid  comparison.  The  appropriate 
interest  rate  is  used  to  discount  or  calculate  future  costs  and  benefits  so 
as  to  arrive  at  their  present  values.  Each  year's  expected  yearly  cost  is 
multiplied  by  its  discount  factor  and  then  summed  over  all  years.  (The 
current  discount  rate  specified  by  OSD  is  10  percent.) 

INDIRECT  COSTS 


An  overhead  or  indirect  cost  factor  is  attached  or  "burdened"  atop 
direct  costs  by  each  contractor  to  account  for  general  support  and  adminis¬ 
trative  expenses.  At  each  Government  or  sponsor  site  used  during  the  life 
cycle  of  the  training  system,  an  indirect  cost  factor  needs  to  be  established. 
If  no  specific  indirect  factor,  per  site,  can  be  determined,  a  "standard" 
service  or  DoD  rate  will  be  used. 

INHERITED  ASSETS 

The  use  of  assets  already  available  (inherited  assets)  requires  careful 
evaluation  when  preparing  a  cost  estimate.  The  fact  that  a  given  system 
component  is  already  available  does  not  automatically  equate  to  zero  cost 
to  the  computer-based  training  system.  Each  inherited  asset  must  be  evaluated 
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on  its  own  merit  and  in  terms  of  whether  its  use  in  connection  with  the 
system  being  costed  will  cause  some  future  expense.  If  so,  that  expense 
must  be  included  in  some  cost  element  in  the  system's  life  cycle.  If  there  — 
will  be  no  future  expense,  the  cost  of  the  item  will  be  included  in  the 
total  life  cycle  cost  but  will  be  highlighted  as  a  sunk  cost.  Such 
existing  assets  will  be  included  at  their  fair  market  value  (as  measured  by 
market  price,  scrap  value,  or  alternative  use)  and  the  basis  for  arriving 
at  the  estimate  will  be  documented. 

INSTRUCTION 

An  organized,  open  information-exchange  process  in  which  the  student 
and  instructional  agent  (human,  programmed  text,  or  intelligent  machine) 
continually  interact.  The  purpose  of  the  interaction  is  to  enable  the 
student  to  reach  some  criterion  of  understanding  or  skill  proficiency  called 
mastery  of  a  given  set  of  objectives.  A  computer  can  aid  this  process 
directly  by  reducing  all  or  portions  of  a  strategy  for  interaction  to  an 
explicit  algorithm  or  set  of  algorithms.  The  computer  can  also  help  by 
providing  accurate,  reliable  massive  storage  and  retrieval  for  records  of 
student  progress. 

INSTRUCTIONAL  SYSTEM  DEVELOPMENT  (ISD) 

"A  systematic  procedure  for  assuring  application  of  instructional 
technology  to  course  planning  and  development . "1  The  five  phases  of  ISD 
are  treated  in  detail  in  the  five  volumes  of: 

2 

Air  Force — "Handbook  for  Designers  of  Instructional  Systems"  ^ 

Army — "Interservice  Procedures  for  Instructional  Systems  Development" 

Navy/Marines — "Interservice  Procedures  for  Instructional  Systems 
Development"^ 

There  is  some  disagreement  between  the  Services  on  hrw  the  Phases  are 
numbered.  In  this  specification,  we  will  adopt  the  Army's  terminology 
of:  Analyze,  Design,  Develop,  Implement  and  Control  to  represent  Phases 
I-V,  respectively. 

An  issue  to  consider  in  working  with  this  specification  is  that  the 
instructional  development  process  does  not  coincide  with  the  computer-based 
training  system  life  cycle  for  hardware  and  software. 


^"Instructional  System  Development,"  Department  of  the  Air  Force,  AFM  50-2, 
31  July  1975,  p.  1.1. 

2AFP  50-58,  15  July  1973. 

3TRAD0C  PAM  350-30,  1  August  1975. 

4NAVEDTRA106-A,  August  1975. 
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The  ISD  Phases  do  not  fit  the  Development,  Procurement,  and  Operation  and 
Maintenance  Phases  of  the  system's  life  cycle  unless  it  is  a  completely  new 
course  of  instruction.  For  most  training,  sections  of  the  course  will  be 
designed,  developed,  tested,  revised,  adapted,  and  integrated  throughout  the 
entire  life  cycle  of  the  computer-based  training  system.  (For  example,  one- 
third  of  Air  Force  instructional  hours  are  revised  each  year  on  the 
average. ) 

One  problem  in  obtaining  cost  data  for  ISD  activities  is  that 
functionally  related  cost  data  are  generally  not  available  from  existing 
records.  That  is,  information  regarding  personnel  time  and  non-personnel 
costs  related  to  such  functions  as  job  analysis,  media  selection,  or 
preparation  of  learning  objectives,  are  generally  not  recorded.  Thus,  such 
data  must  be  collected  during  these  activities  or  projected  by  the  analyst 
who  is  complying  with  this  specification.  One  approach  would  be  to 
designate  an  individual  with  the  responsibility  of  maintaining  a  detailed 
log  of  on-going  instructional  system  development. 

For  purposes  of  this  specification,  the  costs  associated  with  all 
activities  related  to  the  first  three  ISD  Phases  (Analyze,  Design,  and 
Develop)  will  be  documented  as  part  of  the  Development  Phase  of  the  system 
life  cycle.  The  costs  associated  with  the  ISD  activities  of  Phases  IV  and 
V  (Implement  and  Control)  will  be  accounted  for  in  the  Operations  and 
Maintenance  Phase.  No  ISD  activities  will  be  shown  as  part  of  the 
Procurement  Phase — but  rather,  this  Phase  will  involve  only  the  cost  of 
purchasing  the  completed  Instructional  products,  materials  and  programs. 

LIFE  CYCLE/LIFE  CYCLE  COSTS 

The  life-cycle  cost  is  the  total  cost  of  an  item  or  system  over  its 
full  life.  The  computer-based  training  system  life  cycle  is  encompassed 
in  three  phases:  Development,  Procurement,  and  Operation  and  Maintenance. 
These  three  phases  are  of  variable  lengths  depending  upon  each  specific 
system.  Although  the  three  phases  overlap,  for  purposes  of  this  specifica¬ 
tion,  they  will  be  considered  to  occur  sequentially.  The  Development  Phase 
can  take  up  to  6  years,  the  Procurement  Phase  is  shown  as  one  year,  and  the 
Operation  and  Maintenance  Phase  can  be  up  to  8  years.  Although  Procurement 
can  take  more  than  one  year,  this  specification  requires  an  assumption  of 
all  Procurement  activities  occurring  within  the  year  following  development 
of  a  tested  prototype  system.  Also,  although  such  systems  are  usually 
phased  in,  it  is  assumed  in  this  specification  that  Procurement  will  be 
followed  by  instantaneous  operation  of  all  systems  acquired. 

The  specific  years  in  which  each  Phase  occurs  are  to  be  noted,  the 
information  coming  either  from  the  contractor  or  service  monitor.  The 
respondent  needs  to  enter  the  year  in  which  the  system  and  its  components 
are  procured  . 

Life  cycles  are  different  for  various  system  components.  These  life 
times  established  by  the  Economic ;  Analysis  Han^ibook,  2nd  edition.  Department 
of  Defense,  are  as  follows: 


ADP  Equipment  8  year a 

Building*  25  " 

Operating  Equipment  10  " 

Utilities,  Plant*  &  Utility 

Distribution  Systems  25  " 

Weapon/Support  Svstems  Variable 


PAY  AND  ALLOWANCES 

The  cost  of  civilian  and  military  personnel  paid  at  annual  rates  will  be 
gross  pay  In  current  pay  tables,  plus  the  Government's  contribution  tor 
civilian  retirement,  disability,  health,  life  Insurance,  and,  where 
applicable,  social  security  programs.  Factors  to  weight  the  base  pav  of 
civilian  and  military  personnel  will  be  based  on  guidance  from  the  Office 
of  the  Assistant  Secretary  of  Defense  (Comptroller),  "Economic  Cost  of 
Military  and  Civilian  Personnel,"  and  recent  policies  o!  the  Office  of 
Management  and  Budget  regarding  civilian  personnel  pav.  The  latter  will 
be  used  to  weight  the  pay  of  civilian  Government  personnel  who  are  direct  lv 
Involved  In  the  life  cycle  of  the  computer-based  training  system.  The 
following  percentages  of  base  pav  will  be  used  in  computing  the  costs  of 
civilian  personnel  services: 

Retirement  2A.7T 

Health  Insurance  l.1** 
l.lfe  Insurance  .St 

The  military  and  civilian  pav  rates,  as  weighted,  do  not  include  special 
pay,  such  as  flvlng  pay  or  hazardous  duty  pay.  These  costs  must  be  added 
to  the  rates  whenever  they  are  required  by  the  lob  or  location.  If  appro¬ 
priate,  pav  should  be  Increased  to  cover  leave  and  other  benefits  such  as 
the  average  cost  of  sick  leave  taken  and  annual,  holldav  and  other  paid 
leave  accruals,  plus  the  average  Government  contrlbut tons  for  other  benefits. 

SUNK  COST 

A  cost  which  Is  Irrevocably  committed  t o  a  project.  Each  cost  analysis 
will  make  explicit  any  cost  which  Is  sunk  at  the  time  the  analysts  is 
prepared.  All  costs  which  reflect  Irreversible  decisions  will  be  treated 
as  sunk. 
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Part  C 


COMPUTER-BASED  TRAINING  SYSTEM  ELEMENTS: 
DEFINITIONS  AND  COSTS 


In  this  section  you  are  to  provide  costing  information  for  each 
element  of  the  computer-based  training  system.  For  Equipment  or  Facilities, 
four  items  of  information  are  required  for  accurate  costing:  the  type  of 
device  or  unit,  the  unit  cost,  the  number  of  units  or  the  proportion  of  one 
unit  allocated  to  the  system,  and  whether  or  not  the  units  are  to  be 
supplied  by  the  government  (e.g.,  GFE)  or  by  the  contractor.  If  you  are 
a  contractor  and  the  units  are  to  be  provided  by  the  government,  check 
the  GFE  space;  otherwise,  give  the  unit  cost.  Maintenance  costs  should 
be  included  wherever  applicable. 

Definitions  of  all  system  elements  are  provided  on  the  left-hand  pages, 
with  examples  of  how  these  items  are  to  be  costed.  For  Software,  Instructional 
Preparation,  Instructional  Methods/Materials,  and  Acceptance  Tes t /Management . 
units  are  hard  to  define  and  unit  costs  are  difficult  to  obtain  or  are  not 
applicable.  In  these  areas,  you  are  to  provide  as  meaningful  a  description 
of  the  item  and  its  characteristics  as  possible.  Enter  the  total  costs  of 
elements  in  these  categories  in  the  appropriate  spaces.  In  some  cases  you 
will  need  to  delineate  the  personnel  costs  of  certain  elements.  To  assist 
you  in  identifying  these  costs,  Personnel  Cost  Worksheets  accompany  this 
specification. 


sum 
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DEFINITIONS 


EQUIPMENT 

Included  in  this  category  are  all  the  components  of  equipment  related  to  a 
computer-based  training  system  (e.g.,  the  computer  and  its  associated  auxiliary 
memory  requirements,  terminals,  carrels,  auxiliary  audiovisual  devices).  Also 
included  are  local  interface  hardware,  telephone  lines,  special  lines,  satellites, 
receivers,  power  generating  equipment,  associated  test  and  checkout  equipment, 
etc.  In  the  Procurement  Phase,  this  category  also  includes  initial  spares  and 
repair  test  equipment.  Maintenance  costs,  derived  from  factors  such  as  mean  time 
before  failure  and  mean  time  or  cost  to  repair,  should  be  included  in  the  costs 
of  every  piece  of  equipment  maintained . 


EXAMPLE 
1.  EQUIPMENT 

1.3  Auxiliary  Audiovisual  Davicai 


Description  | 

No 

Units 

Unit 
*  Col! 

Total  GFE 

Cost  ( /) 

? 

* 

s  IMS' 

*.  '  . '  “ ' 

/ 

3*iS 

3 

t ■  /LlU ^ rwilH  1 

/ 

*+<TD 

i  a 

4 

a. 

Computer(s) :  This  refers  to  the  hardware,  either  contractor  or  in-house, 
required  to  produce  the  automatic  data  processing  capability  of  the  system.  It 
includes  the  installed  machine  or  group  of  inter-connected  machines  consisting  of 
input,  storage,  computing,  control  and  output  devices  which  use  circuitry  in  the 
main  computing  element  to  automatically  perform  arithmetic  and/or  logical  opera¬ 
tions  by  means  of  internally  stored  or  externally  controlled  programed  instructions. 
This  element  includes,  for  example,  a  central  processor,  large  capacity  storage 
data  channels,  and  input/output. 

Terminal (s) :  Refers  to  the  hardware  required  to  produce  the  data  display 
portion  of  the  system.  It  includes  the  equipment  necessary  to  provide  visual  pre¬ 
sentation  of  processed  data  or  instruction  by  means  of  specifically  designed 
electronic  or  electromechanical  devices  interconnected  with  the  computing/processing 
subsystem,  such  as  flat  panel  displays,  projection  screens,  image  data  storage 
and  retrieval  equipment. 

Auxiliary  Audiovisual  Devices:  This  subcate’gory  consists  of  equipment  which 
presents  audiovisual  instructional  material.  Included  are  such  devices  as: 
slide  projectors,  slide/tape  units,  film  strip  and  overhead  projectors,  various 
movie  film  projectors,  videotape  recorders/players,  etc. 
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1.  EQUIPMENT 


1.1  Computer(s) 


description 


No.  Unit  _  Tout 

Units  Cost  *  Cost 


GFE 

(O 


i _ _  $ _  $ 

b. 


c. 

d. 

«. 


| 

(To  P»H  D  p  61 ) 

1.2  Terminal(s) 


a. 

b. 


c 

d. 


1.3  Auxiliary  Audiovisual  Devices 


$  $ 


$ 


(To  r«it  O  p  611 


$  $ 


Auxiliary  Memory:  Storage  capability  designed  to  extend  the  core  memory 
of  &  computer.  It  may  consist  of  serial  access  devices  (e.g. ,  magnetic  tape) 

or  random  access  (e.g.,  disc  or  drum)  or  possibly  extended  core  storage  (ECS) 
or  shift  registers  added  to  the  main  memory.  The  medium  is  electro-magnetic 
although  paper  tape  could  be  used. 


Local  Interfaces:  Direct  transmission  connectors  linking  various  peri¬ 
pheral  devices  (disc,  tape  drivers,  CRTs,  etc.)  with  the  computer  or  in  some 
cases  with  each  other. 


Telephone  Lines:  Standard  public  commercial  voice  grade  channels 
(generally  up  to  3000  cps)  usable  in  computer-based  training  systems. 


1.4  Auxiliary  Memory 


Description 

No.  Unit  m  Totel 

Units  Cost  Cost 

a 

$  S 

b. 

c. 

d. 

». 

$ 


(To  Part  D  p.  611 


1.5  Local  Interfaces 

«.  _ _ 

b. _ 

c _ 

d. _ 

a 


$  $ 


$ 


(To  Part  D  p.  61 ) 


1.6  Telephone  Lines 
a 


$  $ 


ch4nnel*  deigned  to  handle 

•tructed  to  transit  baud  rateHor  Mr!  ”  P°intm'  Th*  ^culta  Irl  con- 

bludT""^0"5  *’*••  30°-3°OO  ops  fol  voice*ap°!l<*>  K#Uv*nt  t0  v*rlouf»  grade* 
(or  higher)  for  high  speed,  vide-band  dlgltl^™  “P  50*000 


higher  frequencies  than  is  stand^ri  r  ’  transmission  is  typicallv 

«<mv.rt.r.  .t  reception  ^  —  '-.for.  r^S£*}*£« 


"ttj^^'ro0^— /J^‘o—notion.1  .ft.  dr.lglloj  to  c.ptor.  th. 

ssa-sssi  ^ :;:r- 

*"  *“h  ••  — ~™uu£r£xs;:  as,rjfe. 


PtutTr^.:..*"  <«■  -*>«»?  "ZLz:?xz;  zzszr 
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f S**11  enclosures  for  Individual  studv  that 

c«wi!?.rF«auui?  “tr“ctur”  °f  th« 


are  moveable, 
they  would  be 


If 


22 


J 

1 

] 


l.l  I  carrels 


Disruption 

No. 

Units 

Unit 

X  Cost 

a  Tout 

Cost 

«L 

s 

S 

b. 

c. 

d. 

«. 

$ 

(To  Par  I  D  p.  61) 


1.12  Initial  Spares  and  Repair  Test  Equipment 

&  _ _ 

b.  _ 

c.  _ 

d.  _ 

•. 


$  $ 


$ _ 

{To  Pan  0  p.  61) 


1.13  Other  Equipment 


FACILITIES 


Included  in  this  category  are  all  the  physical  facilities  required  for  housing 
the  equipaent  components,  administrators,  and  users  of  the  computer-based  training 
system.  This  category  includes  classrooms,  laboratories,  large  group  instruction 
spaces,  offices.  Individual  learning  spaces,  libraries  and  other  information 
resource  centers,  etc.  Units  should  be  specified  as  the  number  of  square  feet 
required  to  house  all  components  of  the  computer-based  training  system. 


Laboratories:  Places  providing  opportunity  for  observation,  practice,  or 
experimentation.  Included  in  this  category  are  simulated  job  settings  which  can 
be  room-size  or  as  large  as  warehouses,  depending  upon  the  particular  training 
situation. 

Large  Group  Instruction  Spaces:  included  in  this  element  are  such  places 
as  auditoria,  study  halls,  demonstration  rooms,  etc.,  where  large  numbers  of 
students  can  be  trained. 

Offices :  Places  in  which  the  personnel  who  develop,  manage,  administer, 
and  support  the  compuer-based  training  system  perform  their  functions. 
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Individual  Learning  Spaces:  This  cateogry  includes  built-in  carrels  and 
small  cubicles  which  serve  as  spaces  for  Individuals  to  study  or  work  by  them¬ 
selves.  If  such  carrels  are  moveable,  they  would  be  classified  as  Equipment. 


Libraries  and  Other  Information  Resouce  Centers:  Places  in  which  printed 
and  other  forms  of  mediated  Information  is  stored  and  arranged  for  use.  Included 
in  this  element  are  learning  resource  centers,  dial-in  TV  and  other  information 
retrieval  systems,  etc. 
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2.5  Individual  Learning  Spaces 


Description 


a 

b. 

c. 

d. 


2.6  Libraries  and  Other  Information  Resource  Centers 

a  _ _ 

b. _ 

c _ 

d. _  _ 


a 


2.7  Other  Facilities 


SOFTWARE 


This  category  of  the  computer-based  training  system  Includes  systems 
programing  support,  general  applications  programs,  diagnostic  and  checkout 
software,  utility  programs,  etc. 


Systems  Programs:  The  underlying  software  necessary  for  the  control  of  a 
computer  system,  including  operating  systems,  monitors,  executives,  peripheral 
device  interfaces,  etc. 

General  Applications  Programs:  Those  computer  programs  which  are  dependent 
on  data  bases  for  their  use  but  which  can  be  run  on  various  machines  and  have 
usefulness  independent  of  a  specific  course  of  instruction  (e.g.,  SPSS,  BloMed 
Statistical  Packages,  Compilers,  Assemblers  and  Interpreters  such  as  FORTRAN, 
BASIC,  COURSEWRITER,  or  certain  interactive  CAI  functions). 

Diagnostic/Test  Programs:  Special  purpose  programs  used  primarily  to  test 
the  operation  of  computer  system  hardware  components.  Examples  Include 
diagnostics  to  locate  damaged  tape  or  disk  surfaces,  faulty  sections  of  computer 
memory,  or  transmission  errors  on  telecommunicat ion  links.  Diagnostic  programs 
often  operate  independently  of  a  computer's  operating  system  software. 
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1 


I 


V 


3-  SOFTWARE 

3.1  Sy (tarns  Programs 

Description 


a. 


b. 


d. 


3.2  Ganeral  Applications  Programs 


a. 


b. 


c. 


d. 

a. 


3.3  Diagnostic/Test  Programs 


a. 


b. 


Utility  Programs ;  Generalized  data  and  file  manipulation  software  used  to 
create,  copy,  modify  or  delete  machine-readable  data.  Such  utilities  include 
programs  to  copy  tapes,  print  tape  and  disk  files,  convert  files  from  one  data 
representation  (e.g.,  ASCII)  to  another,  etc. 


3.4  Utility  Programs 


Description 

Cost 

1 

1 

a. 

$ 

b. 

c. 

d. 

a. 

3.5  Other  Computer  Programs 


$ 

(To  P#'I  O  l>  6" 


a. 


S 


INSTRUCTIONAL  PREPARATION 


This  category  includes  all  training  costs  necessary  for  initial  site  per¬ 
sonnel  to  acquire  sufficient  skills  to  operate,  maintain,  and  support  the 
computer-based  training  system.  Included  are  the  costs  of  training  initial 
site  personnel  cadre  as  well  as  their  pay  and  allowances  during  the  period  in 
which  they  receive  training. 


Training  of  Initial  Site  Personnel  Cadre:  This  subcategory  refers  to  all 
the  training  services,  courses,  devices,  accessories,  aids,  equipment,  facilities, 
and  parts  used  to  facilitate  instruction  through  which  initial  site  personnel 
will  acquire  the  skills  necessary  to  operate  and  maintain  the  computer-based 
training  system. 

Site  Personnel  Pay  and  Allowances:  This  subcategory  refers  to  the  initial 
site  personnel  costs  (civilian  and  military)  while  they  are  in  a  training  status. 
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$ 

(To  Pan  11  p  til  I 

4.3  Other  Instructional  Preparation  Costs 


INSTRUCTIONAL  METHODS/MATERIALS 

Included  here  are  all  forms  of  instructional  methods,  materials,  and  tests 
in  the  computer-based  training  program.  All  print  and  mediated  instruction  is 
included,  as  well  as  specific  applications  computer  programs  when  appropriate. 
Printed  materials  would  include  such  items  as  training  manuals,  instructor  guides, 
printouts,  books,  programmed  texts,  etc.  Other  mediated  forms  of  instruction 
include  film,  audio,  audiovisual,  video  and  computer  displays,  etc.  Indicate 
the  estimated  number  of  course  hours  for  each  element  of  instruction  procured. 


Audio:  In  this  subcategory  are  included  instructional  materials  in  forms 
presented  only  for  listening.  Tape  recordings,  phonograph  records,  radio 
broadcasts,  etc.,  comprise  this  subcategory. 

Audiovi sual :  Mediated  instructional  materials  which  are  to  be  seen  and 
heard  in  an  integrated  fashion.  Included  in  this  subcategory  are  slide/tape 
programs,  sound  filmstrips,  sound  motion  pictures,  videotape  and  other  television 
programs,  etc. 

Film/Text/Visual :  Mediated  instructional  materials  on  film  designed  to  be 
viewed  only.  Included  in  this  subcategory  are  slides,  overhead  transparancies , 
film  strips,  silent  motion  pictures,  etc.,  in  which  text  as  well  as  pictures 
are  presented. 

Lecture/Demonstration :  A  portion  of  instruction  in  which  facts  or  concepts 
to  be  learned  are  presented  by  the  instructor,  or  a  skill  to  be  mastered  may  be 
introduced.  Active  student  participation  may  be  elicited  at  critical  points  with 
(1)  questions  or  (2)  by  directing  attention  to  certain  features  demonstrated. 
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5.  INSTRUCTIONAL  METHODS/MATERIALS 
5.1  Audio 


5.3  Film/Text/Visual 


(To  P»rt  D  p.  62) 


5.4  Lactura/Damonstration 


(To  Part  D  p.  62) 


a. 


Group  Discussion/Seminar:  A  small  number  of  students  engaged  in  interactive 
exchange  of  information  about  a  given  topic.  The  instructor  may  or  may  not 
participate. 


Performance/Practice :  Exercise  for  the  student  to  rehearse  a  skill  (1)  with 
feedback  (guided  practice),  or  (2)  without  feedback  (unguided  practice)  on  the 
student's  performance  of  the  skill. 


Tutoring  (peer  or  other):  A  form  of  instruction  involving  one-to-one 
dialogue  between  instructional  agent  (teacher,  fellow  student  or  other)  and 
student. 


Printed  Text/Visual:  Instructional  materials  that  are  printed  on  paper. 
Included  in  this  subcategory  are  books,  training  manuals,  programmed  texts, 
printouts,  forms,  as  well  as  printed  photographs,  etc. 
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5.5  Group  Discusston/Seminar 


a. 


b. 


d. 


5.6  Performance/Practice 


b. 


d. 


5.7  Tutoring  (peer  or  other) 


b. 


d. 


5.8  Printed  Text/Visual 


Computer-Administered  Instructional  Materials,  instructional  materials  that 
are  prepared  for  presentation  on  computer  output  devices  such  as  cathode  ray 
tubes,  plasma  panels,  teletypes,  etc.  A  variety  of  instructional  strategies  can 
be  applied  using  the  computer  as  described  below. 

Drill  and  Practice:  Following  presentation  of  a  concept  and  a  test  of 
knowledge  comprehension,  the  trainee  interacts  with  a  series  of  example  instances. 
The  purpose  of  this  activity  is  to  strengthen  performance  skills  in  the  designated 
concept.  Computer-based  drill  and  practice  permits  repeated  presentations  of  a 
wide  range  of  randomly  generated  or  specially  programmed  instances.  Acceleration 
or  remediation  of  instruction  can  be  made  contingent  on  student  performance. 


Sinmlat ion:  An  instructional  situation  which  allows  a  participant  to 
interact  in  a  situation  closely  resembling  an  actual  experience.  Depending  on  the 
sophistication  of  the  computer  system  involved,  the  computer  houses  the  simulation 
rules  and  enables  complex  simulated  decision-making,  monitors  student  progress  and 
maintains  instructional  records  for  subsequent  training  use.  Justification  for 
simulation:  safety  in  performing  dangerous  and  critical  criterial  tasks,  tele¬ 
scoping  real-world  time,  and  permitting  abstraction  of  essential  task  elements 
from  potentially  confusing  total  criterial  environment. 


Games :  A  cooperative  or  competitive  environment  in  which  the  student 

interacts  with  real  or  artificial  participants  to  achieve  specified  goals.  Points 
can  be  offered  for  degree  of  cooperative  or  competitive  achievement.  In  combina¬ 
tion,  simulation  and  games  are  often  used  as  instructional  tools  for  teaching 
problem-solving.  They  are  justified  as  strong  motivating  forces  for  learning. 

In  addition  to  the  automated  functions  ascribed  to  simulation,  the  computer  can 
provide  the  rules  for  real  players  to  interact;  or  it  also  can  be  the  source 
through  artificial  intelligence  of  other  simulated  players. 
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5.9  Computer- Administered  Instructional  Materials 

5.0.1  Drill  and  Practice 


Description 


a. 

b. 

c. 

d. 

a. 


6.9.2  Simulation 


6.9.  S  Gamas 


■  .  *«*  instructional  situation  in  which  th«  train**  interact* 

wltn  a  nodal  of  a  t*ach*r,  the  nodal  being  r**id*nt  in  th*  co^uiter.  It  can 

!11  V  °f  th*  ln,tructlonal  progran,  governing  (dialogue)  sequence 
of  natwiala,  feedback  rules,  strategies  for  simulation,  ganes,  drill  and  practice 
or  other  related  instructional  paradigms.  Depending  upon  conputer  sophistication 
and  instructional  needs,  autonatlon  of  instructional  prescription,  resource 
allocation,  and  record-keeping  can  be  partial  or  total. 


s.  • 


V-  v' 


.  ■'  '  .  v  -  -V 

T  , 
If 

•  ,  l.' 

- ■ "  << 

■  V 


V  .  .  Solving:  One  of  the  most  common  instructional  forms  consists 

of  quantitative  problems  generated  by  stored  algorithms.  The  trainee  Interacts 
by  inputting  step-by-step  process  solutions  at  a  terminal  device  or  th*  trainee 
solves  th*  problem  on  paper  and  inputs  his  answer. 


i 

r-y 


J  - 


Injuirj[:  The  trainee  forms  questions  which  he  addresses  to  the  computer 
system.  He  may  us*  natural  language  or  some  eaay-to-learn  subset  of  the  language. 
The  system  processes  the  questions  usually  using  key  words  and  searching  stored 
algorithms  to  provide  an  answer.  This  activity  is  often  called  Information 
retrieval. 
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Specific  Applications  Programs :  Computer  programs  written  to  satisfy 
a  specific  instructional  logic  requirement  in  a  given  course.  These  programs 
are  both  data  and  course  dependent  (such  as  unique  algorithms  written  for 
Physics  I  problems  for  a  particular  Instructor).  They  are  usually  written  in  a 
CAI  language  although  they  could  also  be  written  in  a  machine  assembly  language. 


Total  Computer-Administered  Instructional  Materials  Costs:  Ehter  total 
costs  obtained  in  subcategories  5.9.1  thru  5.9.8  in  the  spaces  provided.  Sum 
these  costs  to  arrive  at  a  total  cost  of  5.9  subcategory.  Transfer  this  sum 
to  the  appropriate  line  of  the  TCBS  (Part  D) . 
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a.  _ _ _  _ 

k _ _ _ _ 

c. _ 

A _ _ 

s.  _ _ _ 

IM  Odor  CompuMtAdminiitaml  Instructional  MiiwMi 


t 


t 

(Iwlot*) 


*. 

k 


i 


*. 


4 


*. 


I _ 


Total  Computer  Administarad  Instructional  Materials  Cost*  (5.9) 

Drill  A  Practica  (5  9.1)  $ 

Simulation  (5.9.2) 

Gamat  (5.9.3) 

Tutorial  (5.9.4) 

Problem  Solving  (5.9.5) 

Inquiry  (5.9.6) 

Specific  Application*  Program*  (5.9.7) 

Othar  Computer  Administered  Instructional  Materials  (5.9.8) 

Total  5.9 
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(To  IN» t  On  4J' 


T-Cs*s *  *n  t*'e  broadest  sense,  tests  are  suiter lals  presented  to  targeted 
individuals  or  groups  In  order  to  measure  their  status  with  respect  to  a  specified 
doaaln  of  behavior.  The  tests  nay  be  direct,  the  behavior  sampled  is  from  the 
domain,  e.g.,  the  student  must  actually  change  a  tire.  On  the  other  hand,  the 
tests  may  be  Indirect,  e.g.,  the  student  describes  by  paper  and  pencil  how  he 
would  change  a  tire.  In  Instruction,  these  materials  sample  student  performance 
during  or  at  the  end  of  a  learning  sequence.  They  may  be  used  solely  to  diagnose 
current  level  of  achievement  or  they  may  be  used  In  addition  to  prescribe-^ 
sequences  of  instruction.  If  used  strictly  for  diagnosis,  the  test  items  are 
usually  presented  without  feedback  of  correct  answers. 


Paper  and  Pencil:  These  are  verbal  materials.  Generally  these 
materials  form  an  Indirect  measure  of  the  student's  level  of  knowledge  or  compre¬ 
hension.  The  student  writes  answers  to  an  Item  bv  choosing  from  among  alternatives 
constructs  a  short  answer  to  complete  a  sentence,  or  writes  an  essav.  The 
inference  is  made  that  achievement  on  these  items  provides  a  valid  and  reliable 
index  for  predicting  achievement  in  the  criterion  environment  (e.g..  lob 
proficiency).  J 


Performance  Testa:  Materials  are  those  with  which  the  student  has  to 
do  dome thing,  l.e. ,  non-verbal.  They  are  generally  of  two  types:  (1)  intermediate 
or  indirect  In  which  the  student  manipulates  symbols,  patterns,  or  physical 
Items  said  to  be  correlated  with  some  desired  performance  not  readily  measured 
in  a  testing  environment;  or  (2)  Job-sample  (direct)— in  which  the  student  is 
given  portions  of  job-tasks  to  perform.  These  tasks  are  representative,  critical 
to  Job-performance,  or  both. 
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Computer-Supported  Testing:  If  a  test  is  adninistered  at  a  display 
device  connected  to  a  computer  (a  terminal),  the  tost  is  said  to  be  administered 
on-line.  If  It  is  administered  manually  by  paper  and  pencil  or  by  performing 
tasks  on  devices  or  equipment  not  connected  to  the  computer,  the  test  is  said  to 
be  administered  off-line.  The  support  roles  of  the  computer  can  include  any  or 
all  of  the  following:  (a)  to  store  and  retrieve  banks  of  test  items,  allowing 
efficient  generation  of  numerous  alternate  test  forms;  (b)  to  score  and  print 
out  results  of  the  test  administrations;  (c)  to  maintain  testing  records  for 
purposes  of  test  validation  and/or  student  diagnosis  and  prescription;  and  (d)  to 
administer  tests  on-line. 


Total  Test  Development  Costs:  Enter  total  costs  obtained  in  suhcategor ios 
5.11.1  thru  5.11.4  in  the  spaces  provided.  Sum  these  costs  to  arrive  at  a  total 
cost  of  5.11  subcategorv.  Transfer  this  sum  to  the  appropriate  line  of  the 
TCBS  (Part  D) . 
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6.11.3  Computer-Supported  Tatting 


Oascription 


a. 

b. 


c. 


d. 


a. 


6.11.4  Other  Taft* 


a. 


b. 


c. 


d. 


a. 


Total  Tost  Development  Costs  (5.11) 

Paper  &  Pencil  (5.1 1.1) 

Performance  Tests  (5.11.2) 

Computer  Supported  Testing  (5.11.3) 

Other  Tests  (5.1 1.4) 

Total  5.11 
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ACCEPTANCE  TEST/MANAGEMENT 


This  category  includes  costs  of  the  technical  and  business  management 
effort  expended  in  the  process  of  producing,  purchasing,  and  installing  the 
tested,  operational  computer-based  training  system.  It  includes  costs  associated 
with  program  management,  acceptance  testing,  engineering  changes,  site  check¬ 
out/activation,  etc. 


Proqram/Project  Management:  This  subcategory  contains  costs  for  planning, 
directing,  and  controlling  the  production  of  a  system  and  assuring  that  planning 
is  accomplished  by  organizations  responsible  for  the  complementary  functions  of 
logistics  and  maintenance  support,  personnel  training,  operational  testing, 
activation,  or  deployment  of  a  system. 

Acceptance  Test:  This  subcategory  contains  costs  of  system  acceptance  test 
activities  including  cost  of  the  detailed  planning,  conduct,  support,  data 
reduction  and  reports  from  such  testing,  and  all  test  items  which  are  consumed 
in  the  conduct  and  support  of  the  acceptance  test  program. 

Engineering  Changes:  This  subcategory  contains  costs  associated  with 
official  alterations  made  to  a  system  while  it  is  still  in  the  manufacturing 
process  (before  official  acceptance  as  operational). 


6.  ACCEPTANCE  TEST/MANAGEMENT 

8.1  Program/Project  Management 


Description 


Personnel 

Hours 


Non- 

Personnel 

Personnel 

s  Total 

Costs 

Costs 

Costs 

$ 

$ 

$ 

GFE 

/ 


e. 

d. 


i. 


$ _ 

(To  P»rt  D  p.  62) 

6.2  Acceptance  Test 


oi 

b. 


$  $  $ 


c. 

d. 


». 


$ 

(To  Part  D  p.  62) 

6.3  Engineering  Changes 

«. _  $  S  $ 


»|..r«!H'n.?l>!»!"VAC!i,at’'0',:  ThiS  ““^category  includes  conversion  „f  the 

facility  to  achleve^operatlonal  1status^'tOUt *  1"S",U“<ion  *™»  ■>«* 


6.4  Sit*  Checkout/ Activation 


Dtttnption 


& 

b. 

c. 

d. 

t. 


6.6  Other  Direct  Management  Costs 


OTHER  DIRECT  COSTS 


This  category  contains  elements  which  are  not  covered  in  other  categories. 
Costs  related  to  travel,  supplies,  consultants,  contracts  and  subcontracts, 
etc.,  not  identified  otherwise  are  represented  here. 


EXAMPLE 

7.  OTHER  DIRECT  COSTS 
7.4  Contractt/Subcontracti 


'fxtX kIAUlL  (ULv^t^UJ.  Illskajus  •'  (LctifZLnAX - 


jut  AlU 


'idxnir 


Suppl ies:  This  subcategory  consists  of  all  expendable  supplies  directly 

employed  in  the  procurement  of  the  computer-based  training  system. 


Travel :  Included  here  are  all  transportation  costs,  per  diem  costs,  etc., 
required  in  the  procurement  of  the  system. 


Consul tantS :  This  subcategory  includes  costs  associated  with  unique 
personal  services  required  in  the  direct  procurement  of  the  computer-based 
training  system,  arranged  for  by  the  prime  contractor  or  Government  or  other 
sponsor. 
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7.  OTHER  DIRECT  COSTS 

7.1  Supplies 


No.  Unit  m  Tot*l 

Dtscnption 

Units  Cost  Cost 

$  $ 


b. 


c. 

d. 


i. 


< 

do  I’m  I  I)  p  (i.'l 


7.2  Travel 

Oascriptioit 


$ 


$ _ 

(To  I’m l  O  \\  t>.1 


7.3  Consultants 


Contracts/Subcontracts :  This  subcategory  includes  materials  and  services 
to  be  supplied  by  private  firms/institutions/organizations,  required  for  the 
direct  procurement  of  the  computer-based  training  system.  These  are  arranged 
or  contracted  for  by  the  prime  system  contractor,  or  by  the  Government  or  other 
sponsor. 


7.4  Contraets/Subcontracts 


Dwcnpbon 


7.5  Other  Direct  Cost* 


Part  D 

TRAINING  COST  BREAKDOWN  STRUCTURE  (TCBS) 


5 


Part  D 


TRAINING  COST  BREAKDOWN  STRUCTURE  (TCBS) 


In  this  part  of  the  specification,  a  form  is  provided  for  entering 
the  total  costs  of  procuring  the  computer-based  training  system.  All 
Procurement  costs  should  be  shown  as  having  been  spent  at  one  time  (in 
the  first  year  following  the  Development  Phase).  As  noted  in  Part  B, 
General  Costing  Assumptions  and  Definitions,  all  costs  should  be  expressed 
in  constant  dollars  (using  the  present  year  as  the  base  year) .  For 
purposes  of  obtaining  total  Procurement  costs,  the  element  costs  shown  in 
Part  C  are  to  be  summed  for  each  subcategory  and  entered  into  the 
appropriate  spaces  on  this  form.  Total  costs  for  each  category  are  to 
be  calculated  and  entered.  These  total  category  costs  are  then  to  be 
summed  to  arrive  at  the  total  Procurement  costs  of  the  system. 


59 


TRAINING  COST  BREAKDOWN  STRUCTURE  (TCBS) 
PROCUREMENT  PHASE 


YEAR  19 _ 


1.  Equipment 


1.1 

Computarfs)  S _ 

1.8 

Satellites 

$ 

1.2 

Terminal  (s)  _ 

1.9 

Receivers 

1.3 

Auxiliary  AV  Devices 

1.10 

Power  Generating  Equipment 

1.4 

Auxiliary  Memory  _ 

1.11 

Carrels 

1.S 

Local  Interfaces 

1.12 

Initial  Spares  &  Repair  Test  Equipment 

1.6 

Telephone  Lines  _ 

_  1.13 

Other  Equipment 

_ _ 

1.7 

Special  Lines  _ 

TOTAL  EQUIPMENT  COSTS 

2.  Facilities 


2.1 

Classrooms 

$ 

2.5 

Individual  Learning  Spaces 

L 

2.2 

Laboratories 

2.6 

Libraries  &  Other  Info.  Resource  Ctrs. 

— 

2.3 

Large  Group  Instruction  Spaces 

2.7 

Other  Facilities 

— 

Z4 

Offices 

TOTAL  FACILITIES  COSTS 

3.  Software 


3.1 

Systems  Programs 

$ 

3.4  Utility  Programs 

L 

3.2 

General  Applications  Programs 

Oiagnostic/Test  Programs 

3.5  Other  Computing  Programs 

3.3 

TOTAL  SOFTWARE  COSTS 

4.  Instructional  Preparation 

4.1  Tnininf  of  Initial  Site  Personnel  Cadre  $ _  4.3  Other  Instructional  Preparation  Com  $ 

4.2  8te  Pmonnel  Pay  end  Allowances 


TOTAL  INSTRUCTIONAL  PREPARATION 
COSTS  S 


5.  Instructional  Methotfe/Matarials 


5.1  Audio  $ 

5.2  Audiovisual 

5.3  Film/Text/Visual 

5.4  Lecture/Damonstration 

5.5  Group  Oiscussion/Saminar 

5.6  IVformanca/Practica 


8.  Acceptance  Test/ Management 

6.1  Propim/Project  Management  $ 

6.2  Acceptance  Test 

6.3  Engineering  Chan  gas 


7.  Other  Direct  Costs 

(Not  Included  Above) 

7.1  Supplies  $ 

7.2  Travel 

7.3  Consultants 


5.7  Tutoring  (peer  or  othar)  S 

5.8  Printed  Text/Visual 

5.9  CAI  Materials 

5.10  Other  Instructional  Methods/Materials 

5.11  Tests 

TOTAL  INSTRUCTIONAL  METHODS/ 
MATERIALS  COSTS 


6.4  Sits  Checkout/ Activation  $ 

6.5  Other  Direct  Management  Costs 

TOTAL  ACCEPTANCE  TEST/ 

MANAGEMENT  COSTS  $ 


7.4  Contracts/Subcontracts  6 

7.5  Other  Direct  Costs 

TOTAL  OTHER  DIRECT  COSTS  $ 


Total  Procurement  Costs  $ 


NOTE 

COST-EFFECTIVENESS  ANALYSIS 

FOR  PURPOSES  OF  THIS  SPECIFICATION,  THE  TRAINING 
EFFECTIVENESS  DATA,  AND  COST-EFFECTIVENESS  ANALYSES  TO  BE 
CALCULATED  FOR  THE  COMPUTER-BASED  TRAINING  SYSTEM  ARE  TO  BE 
DOCUMENTED  AS  PART  OF  THE  DEVELOPMENT  AND/OR  OPERATION  AND 
MAINTENANCE  PHASES  OF  THE  SYSTEM  LIFE  CYCLE  (VOLUMES  I  £  III), 
AND  ARE  NOT  FOUND  IN  THIS  VOLUME. 
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